Benchmark for Evaluating Biological Image Analysis Tools

Biological images are critical components for a detailed understanding of the structure and functioning of cells and proteins. Image processing and analysis tools increasingly play a significant role in better harvesting this vast amount of data, most of which is currently analyzed manually and qualitatively. A number of image analysis tools have been proposed to automatically extract the image information. As the studies relying on image analysis tools have become widespread, the validation of these methods, in particular, segmentation methods, has become more critical. There have been very few efforts at creating benchmark datasets in the context of cell and tissue imaging, while, there have been successful benchmarks in other fields, such as the Berkeley segmentation dataset [1], the handwritten digit recognition dataset MNIST [2] and face recognition dataset [3, 4]. In the field of biomedical image processing, most of standardized benchmark data sets concentrates on macrobiological images such as mammograms and magnet resonance imaging (MRI) images [5], however, there is still a lack of a standardized dataset for microbiological structures (e.g. cells and tissues) and it is well known in biomedical imaging [5]. We propose a benchmark for biological images to: 1) provide image collections with well defined ground truth; 2) provide image analysis tools and evaluation methods to compare and validate analysis tools. We include a representative dataset of microbiological structures whose scales range from a subcellular level (nm) to a tissue level (μm), inheriting intrinsic challenges in the domain of biomedical image analysis (Fig. 1). The dataset is acquired through two of the main microscopic imaging techniques: transmitted light microscopy and confocal laser scanning microscopy. The analysis tools1in the benchmark are designed to obtain different quantitative measures from the dataset including microtubule tracing, cell segmentation, and retinal layer segmentation.